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DETAILED ACTION 
Response to Arguments 

1 . The amendment filed 5/30/2008 liave been entered and made of record. 

2. The application has pending claim(s) 1-3 and 5-7. 

3. The Applicant's arguments with respect to claims 1 -3 and 5-7 have been 
considered but are moot in view of the new ground(s) of rejection because the Applicant 

has amended independent clalm(s) 1. 

4. Applicant's arguments filed 5/30/2008 have been fully considered but they are 
not persuasive. 

Applicant alleges, "It is clear from the examiner's comments ..." in page 4 
through "The examiner actually admits ..." in page 4, and states respectively that the 
prior art reference Vivarelli doesn't suggest a Bayesian network configured to produce a 
result with evidence missing. Firstly, the Examiner doesn't admit that Vivarelli's network 
is not a Bayesian neural network, the Examiner does believe it is a Bayesian neural 
network as is seen in the title of Vivarelli and throughout the reference of Vivarelli. The 
Examiner only stated that the Vivarelli reference doesn't explicitly state or suggest that 
the Bayesian neural network is configured to produce a result with evidence missing. 
Although the reference doesn't explicitly state that the Bayesian neural network is 
configured to produce a result with evidence missing, it is a known feature of the 
Bayesian network as is even noted by the inventor Jiebo Luo in the Declaration under 
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37 C.F.R. 1 .132 filed 10/09/2007, where Luo clearly states that a conventional Bayesian 
network is trained to be able to perform when data of input information is missing [see 
Declaration, page 1 , Point 7]. Therefore the inventor clearly admits that the amended 
limitation is conventional in that a conventional Bayesian network may perform with 
evidence missing. However, to expedite prosecution, the Examiner has found a new 
reference by the inventor dated two years prior to the filing of the current application 
[Luo, J. - "Indoor vs Outdoor classification of consumer photographs using low-level 
and semantic features" - IEEE, 2001 , pages 745-748, this reference was cited by the 
Examiner in a PTO-892 dated 2/08/2007] which teaches using a Bayesian network 
[instead of a Bayesian trained neural network as discussed in Vivarelli] for scene 
classification (see Luo, paragraph "In this paper, we propose a Bayesian ..." in page 
745, Section 2 in page 746). Therefore, it would have been obvious to one of ordinary 
skill in the art at the time the invention was made to modify Tretter's method using the 
Luo teachings of making a final classification using a combination of both content based 
and meta data based classification through nodal neural analysis (see Tretter, 
paragraph [0009] at lines 1-11, paragraph [0010] at lines 1-5) by replacing it with a 
Bavesian network in order to further improve the accuracy of the classification by 
applying the appropriate probability contribution weights to the two classifiers (Tretter's 
content based and meta-data based classification). Therefore claims 1 -3 and 5-7 are 
still not in condition for allowance and are still not patentably distinguishable over the 
prior art references. Further art rejection discussions are addressed below in the Art 
Rejection section of this Office Action. 
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The Applicant alleges, "Tine examiner further states ..." in page 6 through 
"Finally, as previously noted by applicants ..." in page 6, and states respectively that no 
one had disclosed or suggested using a Bayesian networl< to solve the problem 
addressed by the present invention and that without acknowledging this problem or any 
other that would lead one to consider the use of Bayesian network to classify images as 
claimed, such a proposed modification of Tretter et al can only be based on hindsight 
knowledge. As discussed above, the Luo reference discloses using the Bayesian 
network to improve scene classification and therefore gives motivation to replace 
Tretter's neural network classifier with a Bayesian network in order to improve Tretter's 
image classification. Therefore claims 1-3 and 5-7 are still not in condition for allowance 
and are still not patentably distinguishable over the prior art references. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 1-3 and 5 are rejected under 35 U.S.C. 103(a) as being unpatentable 

over Tretter (US 2002/0140843 Al , as applied in previous Office Action) in view of Luo 
("Indoor vs Outdoor classification of consumer photographs using low-level and 
semantic features" - IEEE, 2001 , pages 745-748, this reference was cited by the 
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Examiner in a PTO-892 dated 2/08/2007 and the article was provided to the Applicant 
with the Non-Final Office Action dated on 2/08/2007). 

Re Claim 1 : Tretter discloses a method for scene classification / content-based and 
metadata based classification (see abstract, lines 1-3) of a digital image comprising the 
steps of (a) extracting one or more pre-determined camera metadata tags / meta data 
from the digital image (see abstract, lines 6-10, paragraph [0008], lines 9-12, meta-data 
may consist of but not limited to an automatic gain setting, film speed, shutter speed, 
white balance, aperture/lens index, focusing distance, date, time, and flash/no flash [at 
least 9 different meta-data's]); (b) obtaining an estimate of image class / meta data 
based classification based on (1) the extracted camera metadata tags / meta data (see 
abstract, lines 1-3, 10-14, paragraph [0008], lines 15-20) and not (2) image content 
features (paragraph [0008] shows that the meta-data classifier only uses data capturing 
attributes such as shutter speed, focusing distance, date and time, flash/no flash), 
thereby providing a metadata-based estimate / meta-data based classification; (c) 
obtaining, separately from the metadata-based estimate (paragraphs [0003]-[0004] and 
[0009] shows that the content based classification only uses content of the subject data 
such as color and contrast) another estimate of image class / content based 
classification of the digital image based on (1 ) image content features / content based 
data (see abstract, lines 1-3, 10-14, paragraph [0008], lines 15-20) and not (2) the 
extracted camera metadata tags (paragraphs [0003]-[0004] and [0009] show that the 
content based classification only uses content of the subject data such as color and 
contrast), thereby providing an image content-based estimate / content based 
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classification; and (d) producing a final estimate of image class / combination of both 
results of the content based classifier and meta data based classifier of the digital image 
based on a combination of the metadata-based estimate / meta data based 
classification and the image content-based estimate / content based classification (see 
abstract, lines 10-14, paragraph [0008], lines 15-20), wherein the combination in step 
(d) is obtained by using a network / nodal neural network configured to produce the 
final estimate of image class with a specific pre-determined set of evidence (see 
paragraph [0032], lines 6-11). 

However, Tretter fails to explicitly suggest that the network is a Bayesian network 
configured to produce the final estimate of image class still capable of producing the 
final estimate of image class if evidence is missing from the pre-determined set of 
evidence. 

Luo discloses that the network is a Bayesian network / Bayesian network 
configured to produce the final estimate of image class / scene classification still 
capable of producing the final estimate of image class if evidence is missing / 
conditional independence relationships from the pre-determined set of evidence (see 
Luo, paragraph "In this paper, we propose a Bayesian ..." in page 745, Section 2 in 
page 746, conditional independence relationships among variables allowing each 
network link be independent of other links to train the entire net of links separately). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Tretter's method using Luo's teachings of 
making a final classification using a combination of both content based and meta data 
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based classification tlirougli nodal neural analysis (see Tretter, paragraph [0009], lines 
1-11, paragraph [0010], lines 1-5) by replacing it with a Bayesian network in order to 
further improve the accuracy of the classification by applying the appropriate probability 
contribution weights to the two classifiers (Tretter's content based and meta-data based 

classification). 

Further, the inventor Jiebo Luo in the Declaration under 37 C.F.R. 1.132 filed 
1 0/09/2007, clearly states that a conventional Bayesian network is trained to be able to 
perform when data of input information is missing [see Declaration, page 1, Point 7]. 
Therefore the inventor clearly admits that the amended limitation is conventional in that 
a conventional Bayesian network may perform with evidence missing. 

Re Claim 2: Tretter further discloses the metadata extracted in step (a) includes one or 

more of exposure time, aperture, shutter speed, brightness value, subject distance / 
focusing distance and flash fired / flash - no flash (see abstract, lines 6-10, paragraph 
[0008], lines 9-12). 

Re Claim 3: Tretter further discloses the image content features in step (c) include one 
or more of color, texture and semantic features (see paragraph [0003], lines 7-8, 
paragraph [0008], lines 1-3, paragraph [0025], lines 6-11). 
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As to claim 5, the claim is the corresponding computer readable medium for 
storing a program claim to claim 1 respectively. The discussions are addressed with 
regard to claim 1 . 

7. Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over Tretter, 
as modified by Luo, and in further view of Schroder et al (US 7,020,330 B2, as applied 
in previous Office Action). The teachings of Tretter as modified by Luo have been 
discussed above. 

However, Tretter as modified by Luo fails to disclose or fairly suggest applying a 
customized image enhancement procedure to the digital image in response to the final 
estimate of image class of the digital image. 

Schroder discloses applying a customized image enhancement procedure / color 

correction process for assigned image class (3) to the digital image in response to the 
final estimate of image class (2) of the digital image (see Fig. 1 , abstract, lines 8-1 1 ). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to further modify Tretter's method, as modified by Luo, by 
using Schroder's teachings by including the application of applying a customized image 
enhancement procedure after image classification in order to apply the color correction 
process which is most likely for a particular image belonging to a particular selected 
image class (see Schroder, col. 3, lines 10-12). 
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8. Claim 7 is rejected under 35 U.S.C. 103(a) as being unpatentable over Tretter as 
modified by Luo and Schroder, and further in view of Cooper ("A novel approach to 
color cast detection and removal in digital images" - SPIE - Jan 2000, vol 3963, pages 
167-177, as applied in previous Office Action). The teachings of Tretter as modified by 
Luo and Schroder have been discussed above. 

However, Tretter as modified by Luo and Schroder fails to disclose or fairly 
suggest the customized image enhancement procedure is color balancing and the 
customized image enhancement procedure includes retaining or boosting brilliant colors 
in images classified as sunset scenes and removing warm-colored cast from indoor 
images classified as tungsten-illuminated scenes. 

Cooper discloses the customized image enhancement procedure is color 
balancing / color cast due to illuminant sources removal and the customized image 
enhancement procedure includes retaining or boosting brilliant colors in images 
classified as sunset scenes / cast removal from outdoor sunset conditions and removing 
warm-colored cast from indoor images / cast removal from indoor natural images 
classified as tungsten-illuminated / unusual illuminant scenes (see title, abstract, line 1, 
section 7 - RESULTS, paragraph 1, lines 5-6, paragraph 2, lines 1-3). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to further modify Tretter's method, as modified by Luo and 
Schroder, using Cooper's teachings by including the color correcting color cast removal 
feature in order to correct the color cast which is encountered in indoor and outdoor 



Application/Control Number: 10/712,657 Page 10 

Art Unit: 2624 

images such as sunset images (see Cooper, section 7 - RESULTS, paragraph 1, lines 
5-6). 



Conclusion 

9. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

1 0. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Bernard Krasnic whose telephone number is (571) 270- 
1357. The examiner can normally be reached on Mon-Thur 8:00am-4:00pm and every 
other Friday 8:00am-3:00pm. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jingge Wu can be reached on (571 ) 272-7429. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Bernard Krasnic 
September 9, 2008 



/Samir A. Ahmed/ 

Supervisory Patent Examiner, Art Unit 2624 



